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LONG-TERM  GOALS 

Make  HYCOM  (HYbrid  Coordinate  Ocean  Model)  a  state  of  the  art  community  ocean  model  with  data 
assimilation  capability  that  can  (1)  be  used  in  a  wide  range  of  ocean-related  research,  (2)  be  used  in  a 
next  generation  eddy-resolving  global  ocean  prediction  system  and  (3)  be  coupled  to  a  variety  of  other 
models,  including  atmospheric,  ice  and  bio-chemical. 

OBJECTIVES 

Collaborative  5-year  (FY00-04)  National  Ocean  Partnership  Program  (NOPP)  project  on  the 
development  and  evaluation  of  HYCOM,  a  scalable  and  data-assimilative  generalized  (hybrid 
isopycnal/terrain-following  (g)/z)  coordinate  ocean  model.  Work  with  collaborators  Eric  Chassignet 
(overall  project  lead  PI)  and  his  group  at  the  University  of  Miami,  Rainer  Bleck  (Los  Alamos  National 
Laboratory),  Ole  Martin  Smedstad  (Planning  Systems,  Inc.),  Carlisle  Thacker  (NO  A  A/Atlantic 
Oceanographic  and  Meteorological  Laboratory)  and  Remy  Baraille  (SHOM).  Apply  HYCOM  to  two 
model  domains,  an  eddy -resolving  Atlantic  domain  (with  ~7  km  resolution  at  mid  latitudes)  and  a 
coarser  resolution  global  domain. 

APPROACH 

This  includes  many  aspects  that  will  be  performed  by  or  in  collaboration  with  consortium  partners  and 
partnering  projects  (five  at  NRL  in  FY01). 

1.  Ocean  model  design:  HYCOM  is  a  generalized  (hybrid  isopycnal/a/z)  coordinate  ocean  model.  It  is 
isopycnal  in  the  open  stratified  ocean,  but  reverts  to  a  terrain-following  coordinate  in  shallow  coastal 
regions,  and  to  z-level  coordinates  near  the  surface  in  the  mixed  layer.  This  generalized  vertical 
coordinate  approach  is  dynamic  in  space  and  time  via  the  layered  continuity  equation,  which  allows  a 
dynamical  transition  between  the  coordinate  types.  Like  MICOM,  HYCOM  allows  isopycnals 
intersecting  sloping  topography  by  allowing  zero  thickness  layers.  HYCOM  was  developed  from 
MICOM  using  the  theoretical  foundation  for  implementing  a  hybrid  coordinate  system  set  forth  in 
Bleck  and  Boudra  (1981)  and  Bleck  and  Benjamin  (1993). 

2.  Model  development:  HYCOM  development  is  a  close  collaboration  between  Los  Alamos  (Rainer 
Bleck),  NRL  (Alan  Wallcraft)  and  the  University  of  Miami  (George  Halliwell),  where  the  person  in 
parenthesis  is  the  lead  performer  in  each  group.  Alan  Wallcraft  is  in  charge  of  developing  and 
maintaining  the  standard  version  of  the  model,  one  that  is  scalable/portable  and  can  run  on  the  latest 
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computer  architectures.  Part  of  this  work  will  be  performed  under  a  partnering  Common  HPC  Software 
Support  Initiative  (CHSSI)  project.  HYCOM  will  be  maintained  as  a  single  source  code  with  the 
maximum  feasible  backward  compatibility.  Salient  issues  include  the  mixed  layer  and  diapycnal 
mixing,  the  hybrid  coordinate  generator,  developing  a  portable  and  scalable  computer  code,  the 
capability  to  run  HYCOM  in  MICOM-mode,  the  ability  to  initialize  HYCOM  from  MICOM-mode  or 
a  true  MICOM  simulation,  and  diagnostic/visualization  capabilities.  A  grid  nesting  capability  is  under 
development  in  a  collaboration  between  U.  Miami  and  a  partnering  project  at  NRL,  the  6.2  Global 
HYCOM  and  Advanced  Data  Assimilation  project  sponsored  by  ONR  (see  separate  report). 

3.  Ocean  modeling  applications:  HYCOM  will  be  applied  to  the  Atlantic  Ocean  (27°S-70°N)  in  a  close 
collaboration  with  Eric  Chassignet’s  group  and  the  global  domain  in  collaboration  with  Rainer  Bleck 
and  Eric  Chassignet.  HYCOM  will  be  tested  on  a  1/3°  Atlantic  model  grid  consistent  with  the 
DYNAMO  Experiment  before  going  to  high  resolution.  The  HYCOM  results  will  be  compared  to 
simulations  run  in  MICOM  mode.  The  high  resolution  grid  is  .08°  cos  9  in  latitude  (0)  by  .08°  in 
longitude  or  ~7  km  resolution  for  each  variable  at  mid-latitudes.  The  high  resolution  Atlantic  modeling 
will  be  performed  mainly  using  a  large  grant  of  computer  time  provided  by  an  approved  FY02-04  DoD 
High  Performance  Computing  (HPC)  Challenge  project  with  Eric  Chassignet  as  the  lead  PI.  NRL 
participation  in  the  global  modeling  will  begin  in  FY02  with  .72°  equatorial  resolution.  Later  it  will  be 
increased  to  .24°  and  in  FY04  to  .08°  in  collaboration  with  partnering  projects.  Atmospheric  forcing 
(wind  and  thermal)  will  be  used  from  both  the  European  Centre  for  Medium  Range  Weather  Forecasts 
(ECMWF)  and  the  National  Centers  for  Environmental  Prediction  (NCEP).  A  wide  range  of  data  sets 
are  available  for  model  evaluation  (Chassignet  et  al.,  2000;  Hurlburt  and  Hogan,  2000)  and  these 
papers  discuss  a  wide  range  of  climatological  model-data  comparisons.  In  addition,  we  have  long  time 
series  of  transports  through  the  Florida  Straits,  sea  level  at  tide  gauges,  and  SST  and  subsurface 
temperature  from  moored  buoys;  also  altimetric  sea  surface  height  (SSH)  and  IR  SSTs  from  satellites, 
PALACE  float  and  BT  data  and  data  from  research  field  programs. 

4.  Data  Assimilation:  The  HYCOM/NOPP  data  assimilation  effort  is  mostly  performed  by  consortium 
partners  outside  NRL:  Ole  Martin  Smedstad  (Planning  Systems,  Inc.,  co-located  at  Stennis),  Carlisle 
Thacker  (NOAA/AOML),  Arthur  Mariano  (U.  Miami),  Mike  Chin  (JPL/U.  Miami)  and  Remy  Baraille 
in  France.  Geir  Evensen  at  the  Nansen  Center  in  Norway  (ensemble  Kalman  filter  method)  and  Pierre 
Brasseur  in  France  (SEEK  filter  method)  are  also  major  contributors  to  the  HYCOM  data  assimilation 
effort.  Under  a  partnering  project,  6.2  HYCOM  and  Advanced  Data  Assimilation,  Greg  Jacobs  (NRL) 
and  Hans  Ngodock  (U.  Southern  Mississippi  and  former  postdoc  of  Andrew  Bennett)  plan  to  work  on 
HYCOM  data  assimilation  using  the  representer  method  and  ensemble  Kalman  filtering,  the  latter  in 
collaboration  with  Geir  Evensen.  The  approach  is  to  implement  a  hierarchy  of  data  assimilation 
techniques  starting  with  relatively  simple  and  efficient  schemes  with  incremental  updating.  The  first 
(Ole  Martin  Smedstad  lead)  will  be  an  Ol-based  scheme  similar  to  that  used  for  the  NRL  Layered 
Ocean  Model  (NLOM).  The  more  advanced  data  assimilation  techniques  will  be  evaluated  against  this 
baseline.  Parameter  Matrix  Objective  Analysis  (PMOA)  will  be  investigated  as  a  potential  upgrade  to 
the  standard  OI  scheme.  PMOA  is  an  OI  scheme  which  is  a  data-driven  analogue  of  a  Kalman  filter, 
one  that  uses  an  assumed  horizontal  covariance  model  with  nine  adjustable  parameters  that  can  vary  in 
space  and  time  (Mariano  and  Brown,  1992).  The  singular  evolutive  extended  Kalman  filter  (SEEK, 
Pham  et  al.,  1998)  and  a  reduced  order  information  filter  (ROIF,  Chin  et  al.,  1999)  will  also  be 
implemented.  Different  vertical  projections  of  the  surface  information  to  the  deep  ocean  will  be 
evaluated  for  the  assimilation  schemes  that  need  them,  e.g.,  Carnes  et  al.  (1990),  Hurlburt  et  al.  (1990), 
Cooper  and  Haines  (1996)  and  Gavart  and  DeMey  (1996).  The  techniques  will  be  evaluated  for 
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computational  efficiency  and  accuracy.  Complete  altimeter  data  sets  from  TOPEX/POSEIDON,  ERS- 
2,  GFO,  JASON- 1  and  ENVISAT  with  additional  corrections  will  be  available  up  to  real  time  via 
NRL;  similarly  for  MCSSTs.  In  addition,  1/8°  mesoscale  depicting  MOD  AS  analyses  of  SSH  and  SST 
with  estimated  error  fields  will  be  available.  The  data  assimilative  model  results  will  be  compared  with 
unassimilated  data  (see  #3  above). 

WORK  COMPLETED 

Two  meetings  of  the  HYCOM/NOPP  partnership  were  held  (Oct.  2000  and  March  2001)  to  review 
progress  and  more  sharply  define  the  plans  and  milestones,  technical  issues,  and  responsibilities  of  the 
participants. 

Model  development  and  basic  testing:  Alan  Wallcraft  completed  development  of  HYCOM  2.0  with  2- 
level  parallelization,  MPI  and  OpenMP,  and  FORTAN  90  coding  style.  It  was  released  for  general  use 
on  3  July  2001.  The  mixed  layer  options  are  KPP  (Large  et  al.,  1997)  and  Kraus-Tumer  and  the  model 
can  be  run  in  MICOM  model  as  well  as  HYCOM  mode.  Alan  developed  the  capability  to  convert  a 
MICOM-mode  or  a  true  MICOM  simulation  (with  layer  1  as  a  bulk-mixed  layer)  to  initialize  a 
HYCOM  simulation  with  layers  added  near  the  surface. 

Dan  Moore  (visiting  scientist  from  Imperial  College,  London)  tested  KPP  in  HYCOM  configured  as  a 
one-dimensional  model  against  Mellor-Yamada  as  implemented  in  the  Navy  Coastal  Ocean  Model 
(NCOM).  He  also  performed  tests  using  an  x-z  plane  upwelling/downwelling  model.  The  model 
includes  shelf  regions  on  both  sides  and  was  forced  by  a  uniform  northward  wind  stress.  These  tests 
were  performed  by  running  HYCOM  with  different  vertical  coordinates  and  resolutions:  pure  z-level, 
pure  a,  hybrid  a-z  and  hybrid  isopycnal-a-z.  The  horizontal  resolutions  ranged  from  2-10  km  in  the 
horizontal  and  from  15-50  coordinate  surfaces  in  the  vertical.  The  results  were  compared  with 
corresponding  results  in  Martin  (1998)  for  the  Princeton  Ocean  Model  (POM)  and  SZM  (the 
forerunner  of  NCOM). 

Model  applications:  Pat  Hogan  ran  several  .32°  Atlantic  simulations  for  10  years  or  more  starting  from 
climatology  or  a  previous  run,  a  few  in  MICOM  mode  for  comparison  to  HYCOM.  One  was  run  1979- 
2000  with  6  hourly  atmospheric  forcing  from  ECMWF.  The  1/12°  HYCOM  Atlantic  modeling  is  a 
major  component  of  the  HYCOM/NOPP  project  and  is  a  collaborative  effort  between  NRL  and  U. 
Miami.  In  support  of  this  effort  an  FY02-04  DoD  High  Performance  Computing  (HPC)  Challenge 
project  was  obtained  with  Eric  Chassignet  as  the  lead  PI.  In  preparation,  the  first  1/12°  Atlantic 
HYCOM  simulation  was  initialized  from  Eric’s  best  1/12°  MICOM  simulation,  which  has  a  very 
realistic  Gulf  Stream  pathway  between  Cape  Hatteras  and  the  Grand  Banks.  This  test  simulation  was 
run  for  74  days  after  intialization  from  MICOM  and  is  illustrated  in  the  figure.  Work  is  underway  on 
appropriate  sampling  of  both  the  HYCOM  and  MICOM  simulations. 

Data  assimilation:  Pat  Hogan  (NRL)  prepared  a  tar  file  with  HYCOM  2.0  bundled  with  everything 
required  to  continue  one  of  his  .32°  Atlantic  simulations  and  use  it  in  data  assimilation  and  testing. 

This  file  was  provided  to  Remy  Baraille  in  France,  Mike  Chin  at  JPL/U.  Miami  and  Ole  Martin 
Smedstad  at  PSI-Stennis. 


3 


Figure  caption:  Snapshots  of  sea  surface  height  (SSH)  and  sea  surface  temperature  (SST)  from  a 
1/12  °HY COM  Atlantic  Ocean  simulation  for  29  August,  68  days  after  initialization  from  MICOM. 

The  simulation  has  ~7  km  mid-latitude  resolution  and  was  atmospherically  forced  by  a  monthly 
mean  ECMWF  reanalysis  climatology  plus  relaxation  to  an  oceanic  climatology  at  the  northern  and 
southern  boundaries  to  include  the  global  thermohaline  circulation. 


RESULTS 

The  collaborative  HYCOM  effort  including  the  HYCOM/NOPP  consortium,  partnering  projects  and 
the  broader  international  community  effort  is  working  extremely  well. 

Model  development  and  basic  testing:  At  the  beginning  of  FY01  HYCOM  1.0  was  a  limping  new 
model  with  limited  parallelization  via  OpenMP.  With  the  release  of  HYCOM  2.0  it  has  reached  the 
maturity  required  for  broad  application.  It  is  scalable  up  to  0(1000  cpus)  via  two  levels  of 
parallelization,  either  or  both  of  which  can  be  used,  and  it  can  run  on  computing  platforms  ranging 
from  PCs  to  a  variety  of  supercomputers  with  different  parallel  architectures.  This  was  accomplished 
using  a  single  source  code  for  all  machine  types.  HYCOM  2.0  has  MICOM  compatibility,  an  asset  in 
comparing  the  two  models  and  initializing  HYCOM  from  existing  MICOM  simulations. 

In  testing  HYCOM,  Dan  Moore  found  that  KPP  in  HYCOM  gave  results  very  similar  to  Mellor- 
Yamada  in  NCOM  or  POM  when  the  KPP  parameter  values  were  appropriately  chosen  (although  these 
are  not  necessarily  the  values  that  would  be  chosen  for  a  typical  HYCOM  simulation).  KPP  is  a  full 
water  column  diapycnal  mixing/diffusion  scheme,  a  particularly  valuable  asset  for  an  ocean  model 
which  includes  isopycnals  as  a  vertical  coordinate.  Apriori,  we  were  concerned  that  HYCOM  might 
not  perform  as  well  in  a  and  z-coordinate  mode  as  models  designed  specifically  for  these  coordinates. 
However,  HYCOM  performed  very  well  and  robustly  using  these  coordinates  in  comparison  to  NCOM 
and  POM.  As  an  advantage,  it  avoids  stairstep  topography  in  z-coordinate  mode  and  is  more  flexible  in 
hybrid  a-z  coordinate  mode  than  NCOM. 

Model  applications:  The  .32°  HYCOM  results  compare  favorably  with  those  from  MICOM  and  even 
show  a  Gulf  Stream  inertial  segment  with  the  correct  separation  at  Cape  Hatteras  much  of  the  time,  a 


4 


capability  not  seen  in  the  corresponding  MICOM  mode  simulation.  The  1/12°  HYCOM  Atlantic 
modeling  is  the  central  high  resolution  modeling  component  of  the  HYCOM/NOPP  project.  The  ability 
to  initialize  HYCOM  from  Eric  Chassignet’s  best  MICOM  simulation  and  the  short  74-day  1/12° 
Atlantic  HYCOM  test  run  are  a  valuable  head  start  on  the  FY02-04  DoD  HPC  Challenge  project  which 
will  provide  the  computer  time  required  for  1/12°  HYCOM  Atlantic  modeling.  The  HYCOM  and 
MICOM  sampling  is  designed  for  HYCOM  and  MICOM  comparison  as  well  as  comparisons  to 
observations.  Particular  emphasis  is  placed  on  comparisons  relevant  to  the  Gulf  Stream,  which 
MICOM  simulated  well,  but  which  is  difficult  to  simulate  accurately  even  at  ~7  km  resolution.  If  the 
HYCOM  simulation  does  not  perform  as  well,  the  comparisons  should  help  us  diagnose  and  correct  the 
problems  that  arise. 

Data  assimilation:  Using  the  HYCOM  2.0/.320  HYCOM  Atlantic  tar  file  provided  by  Pat  Hogan, 

Remy  Baraille  has  already  implemented  the  Cooper  and  Haines  (1996)  technique  for  downward 
projection  of  sea  surface  height  data  into  HYCOM  and  he  has  provided  it  to  HYCOM/NOPP  partner 
Ole  Martin  Smedstad.  With  the  addition  of  this  component.  Ole  Martin  has  everything  needed  to 
develop  a  baseline  HYCOM  data  assimilation  capability  for  satellite  altimeter  data. 

IMPACT/APPLICATIONS 

HYCOM  is  designed  to  make  optimal  use  of  three  types  of  vertical  coordinate,  isopycnal,  a  and  z- 
level.  Isopycnals  are  the  natural  coordinate  in  stratified  deep  water,  terrain-following  (a)  coordinates  in 
shallow  water  and  z-levels  in  the  mixed  layer.  The  layered  continuity  equation  allows  a  dynamical 
space  and  time  varying  transition  between  the  three  coordinate  types.  HYCOM  also  permits  isopycnals 
intersecting  sloping  topography  by  allowing  zero  thickness  layers.  Therefore,  it  should  allow  accurate 
transition  between  deep  and  shallow  water,  historically  a  very  difficult  problem  for  ocean  models.  It 
also  allows  high  vertical  resolution  where  it  is  most  needed,  over  the  shelf  and  in  the  mixed  layer.  The 
isopycnal  coordinate  reduces  the  need  for  high  vertical  resolution  in  deep  water.  HYCOM  is  a  new  and 
promising  design  for  next  generation  global  and  regional  ocean  prediction  systems  and  it  extends  the 
range  of  application  for  ocean  models  in  research.  The  NRL  PI  and  Eric  Chassignet  (U.  Miami)  are 
members  of  the  U.S.  and  International  Steering  Teams  for  the  Global  Ocean  Data  Assimilation 
Experiment  (GODAE),  a  multinational  project  designed  to  help  justify  a  permanent  global  ocean 
observing  system  by  demonstrating  useful  near  real-time  ocean  products. 

TRANSITIONS 

None. 

RELATED  PROJECTS 

The  HYCOM/NOPP  consortium  includes  E.P.  Chassignet  (Coordinator),  A.  Mariano,  G.  Halliwell, 
and  T.M.  Chin  (JPL/U.  Miami),  R.  Bleck  (LANL),  H.  Hurlburt,  A.  Wallcraft,  P.  Hogan,  R.  Rhodes, 
and  G.  Jacobs  (Naval  Research  Laboratory),  O.  M.  Smedstad  (Planning  Systems,  Inc.),  W.C.  Thacker 
(NOAA/AOML)  and  R.  Baraille  (SHOM).  Partnering  projects  at  NRL  include  an  NRL  6.1  ONR  JES 
DRI  project,  6.1  LINKS,  6.1  Thermodynamic  and  Topographic  Forcing  in  Global  Ocean  Models,  6.2 
Global  HYCOM  and  Advanced  Data  Assimilation,  6.3  High  Fidelity  Simulation  of  Littoral 
Environments  (CHSSI))  and  6.4  ADFC  Support.  Additionally,  the  project  receives  grants  of  HPC  time 
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from  the  DoD  High  Performance  Computing  Modernization  Office.  The  NRL  PI  is  a  member  of  both 
the  International  and  U.S.  GODAE  Steering  Teams. 
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